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FOREWORD 


The  Fort  Leavenworth  Field  Unit  of  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  supports 
the  Combined  Arms  Center  with  research  and  development  in  com¬ 
bined  arms  operations  and  command  group  training.  Measurement  of 
staff  performance  is  an  issue  common  to  research  in  both  opera¬ 
tions  and  training.  Performance  assessment  is  also  essential  to 
command  group  training  to  provide  diagnostic  feedback  to  the 
training  audience.  However,  no  good  techniques  have  been  avail¬ 
able  for  measuring  performance  in  this  complex  arena.  To  help 
fill  this  gap,  the  ARI  Fort  Leavenworth  Field  Unit  undertook  a 
developmental  effort  to  provide  a  staff  performance  measurement 
tool  for  use  at  division  and  corps  levels. 

Work  on  this  project  has  been  conducted  under  a  memorandum 
of  agreement  (MOA)  with  Combined  Arms  Center,  Department  of  the 
Army  (CACDA)-C3I:  Development  of  Performance  Measurement  Method¬ 
ology  for  Corps,  Division,  and  Brigade  Command  Posts,  dated  July 
1986;  and  a  MOA  with  CATA:  Development,  Maintenance,  and  Utili¬ 
zation  of  a  Data  Base  Containing  Corps  and  Division  Training  Per¬ 
formance  and  Battle  Event  Data,  dated  March  1989.  To  date,  this 
measurement  tool  has  been  applied  at  command  group  exercises  con¬ 
ducted  by  six  divisions  and  one  corps.  This  report  examines  the 
results  of  the  division  applications  to  determine  if  patterns  or 
trends  can  be  detected  and  to  evaluate  the  utility  of  this  mea¬ 
surement  tool  for  command  group  performance  assessment. 


EDGAR  M.  JOHNSON 
Technical  Director 
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EXECUTIVE  SUMMARY 


Requirement: 

The  Army  Command  and  Control  Evaluation  System  (ACCES)  was 
developed  as  a  performance  measurement  tool  for  division  and 
corps  command  and  control.  To  date,  ACCES  has  been  applied  to 
one  corps  and  six  division  exercises.  The  use  of  ACCES  data  has 
previously  been  limited  to  providing  feedback  to  the  exercising 
units.  The  purpose  of  the  project  reported  here  was  to  look  at 
the  data  across  all  division  applications  to  determine  if  pat¬ 
terns  or  trends  emerged  that  may  be  significant  to  the  whole 
Army.  Additionally,  the  present  project  assessed  ACCES  measures 
so  that  improvements  can  be  made  in  future  applications. 


Procedures : 

Data  from  all  six  division  exercises  were  used  in  the  analy¬ 
sis.  The  decision  was  made  to  eliminate  the  corps  data  from  the 
analysis  because  many  of  the  corps  scores  differed  substantially 
from  those  of  the  divisions.  Due  to  the  exploratory  nature  of 
the  early  ACCES  applications,  measures  were  frequently  revised 
and  redefined.  Of  the  43  measures  that  an  ACCES  application 
yields,  11  measures  were  found  and  collected  and  reduced  in  a 
consistent  fashion  across  all  applications.  An  additiona  nine 
measures  were  found  and  collected  consistently  across  the  last 
five  applications.  Daily  and  overall  scores  for  each  of  these  20 
measures  were  extracted  from  the  final  reports  produced  for  each 
exercising  unit.  The  analysis  of  these  data  focused  on  uncover¬ 
ing  trends  in  the  scores  and  on  the  measurement  quality  of  the 
measures  themselves.  Because  of  the  small  number  of  applications 
available,  the  statistical  analyses  were  mostly  descriptive  in 
nature. 


Findings: 

The  data  obtained  from  these  ACCES  applications  have  pro¬ 
vided  preliminary  insights  into  the  general  strengths  and  weak¬ 
nesses  in  command  and  staff  processes.  The  divisions  studied 
performed  well  in  their  ability  to  involve  multiple  staff  members 
in  the  formulation  of  clear  and  comprehensible  plans  in  accor¬ 
dance  with  their  commander's  guidance  and  to  disseminate  those 
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plans  in  a  timely  manner.  Their  performance  was  relatively 
weaker  in  the  areas  of  thoroughness  of  briefings  and  accuracy 
with  which  forces  were  plotted  on  command  post  maps. 


The  most  debilitating  problem  found  with  the  ACCES  measures 
is  the  reliance  upon  percentage  figures  rather  than  base  rate 
information.  Additionally,  several  measures  were  found  to  per¬ 
form  unsatisfactorily.  Correcting  these  defects  will  be  the 
focus  of  future  work  efforts. 


Utilization  of  Findings: 

This  review  has  found  evidence  of  a  command  and  control 
measurement  methodology  that  shows  great  promise  for  becoming 
highly  useful,  not  only  as  a  measurement  tool  for  command  and 
control  training,  but  as  a  means  of  evaluating  physical  and 
operational  changes  in  the  command  post  environment.  Addi¬ 
tionally,  after  the  data  base  of  ACCES  applications  has  grown 
somewhat,  this  information  will  be  useful  in  further  development 
of  lessons  learned  from  division  and  corps  training  exercises. 
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Introduction 

The  command  and  control  (C2)  process  is  widely  accepted  as  an 
essential  ingredient  in  battlefield  success,  but  it  is  a  very 
difficult  area  in  which  to  measure  performance,  especially  at 
division  level  and  above.  Over  the  years,  a  number  of  methods 
have  been  developed  to  measure  C2  performance  at  various  eche¬ 
lons.  These  methods  have  been  applied  with  limited  success  (see 
Garlinger  and  Fallesen,  1988,  and  Crumley,  1989).  To  fill  this 
measurement  gap,  especially  for  divisions,  the  Fort  Leavenworth 
Field  Unit  of  the  U.S.  Army  Research  Institute  undertook  a  devel¬ 
opmental  effort  that  resulted  in  the  Army  Command  and  Control 
Evaluation  System  (ACCES) . 

The  primary  focus  of  ACCES  is  on  the  overall  performance  of 
the  division  headquarters  at  various  stages  of  the  planning 
process,  from  the  collection  of  information  through  the  develop¬ 
ment  and  implementation  of  plan.;.  The  ACCES  methodology  treats 
the  headquarters  as  an  adaptive  system  that  operates  in  control 
cycles  in  order  to  keep  selected  battlefield  elements  within 
expected  limits.  The  general  approach  is  illustrated  in  Figure 
1.  It  assumes  that  the  headquarters  performs  a  number  of  proc¬ 
esses  in  order  to  support  decisionmaking  and  implementation. 
Additionally,  in  order  to  provide  a  complete  evaluation  of  com¬ 
mand  and  control,  ACCES  also  looks  at  the  performance  of  the 
individual  functional  cells  within  a  division  headquarters,  as 
well  as  the  interactions  among  functional  cells.  Individual 
ACCES  measures  will  be  discussed  in  later  sections  of  this  paper. 

To  date,  ACCES  has  been  applied  in  exercises  conducted  by  six 
divisions  and  one  corps.  Thus  far,  use  of  the  data  generated 
during  ACCES  applications  has  been  limited  to  providing  perfor¬ 
mance  feedback  to  the  exercising  units.  The  purpose  of  the 
project  reported  here  was  to  look  at  the  data  across  the  set  of 
division  applications  in  order  to  determine  patterns  or  trends 
that  may  have  significance  to  the  whole  Army.  The  following 
sections  of  this  report,  will  describe  the  ACCES  measures  analyzed 
and  the  analysis  process  used.  C2  lessons  learned  that  have 
emerged  from  this  analysis  will  be  discussed,  as  well  as  weak¬ 
nesses  found  in  the  ACCES  measures  themselves.  The  report  will 
conclude  with  recommendations  for  future  work  to  improve  ACCES 
in  order  to  1)  provide  better  C2  performance  feedback  to  units, 
and  2)  continue  development  of  a  tool  for  examination  of 
the  C2  process. 
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EXTERNAL 


The  command  post  as  an  adaptive  control  system. 
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Procedures 


Exercise  Data  Base 


There  have  been  seven  applications  of  the  ACCES  methodology  to 
Army  training  exercises  since  late  1986.  The  first  six  of  these 
were  division  exercises  and  the  last  was  a  corps  exercise.  The 
results  of  the  division  applications  were  reviewed  during  our 
analysis . 

The  first  two  division  applications  were  performed  in  conjunc¬ 
tion  with  an  Operational  Test  and  Evaluation  Agency  (OTEA)  field 
evaluation.  At  this  time,  the  ACCES  methodolody  was  still  called 
HEAT  (Headquarters  Effectiveness  Assessment  Tool)  under  which  ti¬ 
tle  it  had  previously  been  applied  in  Naval  Fleet  exercises.  The 
last  four  of  the  five  applications  were  performed  in  conjunction 
with  WarFighter  exercises  conducted  by  the  Battle  Command 
Training  Program  (BCTP)  headquartered  at  Fort  Leavenworth.  One 
application  was  in  conjunction  with  a  division  CPX  driven  by  the 
First  Battle  BC  wargame.  The  BCTP  exercises  are  intense, 
computer-supported,  around-the-clock  exercises  typically  lasting 
five  days  and  intended  to  exercise  all  aspects  of  division  or 
corps  command  post  operations  plus  those  of  its  major  subordinate 
commands.  The  First  Battle  BC  exercise  is  quite  similar  in 
duration  and  intensity,  but  lacks  the  computer  support  to  the 
exercise  controllers.  The  six  divisions  analyzed  represent  light 
infantry,  mechanized  infantry  and  armored  divisions  executing 
some  variation  of  their  primary  missions. 

Method 


The  primary  objective  over  these  seven  applications  has  been 
to  adapt,  improve,  and  streamline  the  ACCES  methodology.  Thus 
the  measures,  the  data  collection  procedures,  and  the  analysis 
techniques  have  not  been  uniform  over  the  seven  applications  (see 
Crumley,  1988) .  Therefore,  our  first  task  was  to  determine  which 
of  the  43  ACCES  measures  were  collected  over  all  seven 
applications  and  calculated  in  a  consistent  fashion.  We  found 
eleven  measures  which  fit  these  criteria  and  we  included  nine 
others  which  were  handled  consistently  over  the  last  five 
applications.  These  20  measures  are  described  in  the  next 
section. 

We  later  decided  not  to  use  the  corps  application  as  the 
scores  were  sufficiently  different  from  those  of  the  divisions  to 
at  least  suggest  a  fundamental  effect  of  echelon  on  the  measures. 
Thus  we  ended  up  with  20  measures  over  four  to  six  applications. 

We  extracted  the  daily  and  overall  scores  for  each  of  these 
measures  from  the  final  reports  produced  for  each  exercising 
unit.  The  foci  of  our  subsequent  analyses  were  to  look  for 
trends  in  the  scores  and  for  those  measures  which  best  discrim¬ 
inated  among  the  divisions.  We  also  looked  for  inconsistencies 
among  the  measures  which  might  indicate  problems  in  application 
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or  interpretation  of  the  measures.  Because  of  the  small  number 
of  applications  available,  little  statistical  analysis  was  per¬ 
formed  on  these  data. 


Measures  Analyzed 

The  43  measures  available  in  the  current  ACCES  are  divided  in¬ 
to  seven  categories  generally  corresponding  to  the  decision  cycle 
steps  which  were  shown  in  Figure  1.  The  category  called  Overall 
Effectiveness  includes  those  measures  which  directly  assess  the 
effectiveness  of  the  command  and  control  system  as  an  adaptive 
coping  system:  whether  its  plans  stayed  intact  or  had  to  be 
modified,  and  the  magnitude  of  changes,  if  required. 

We  were  able  to  use  four  of  the  five  measures  of  overall  ef¬ 
fectiveness,  all  measures  of  the  assessing  and  understanding 
step,  and  six  of  the  eight  measures  dealing  with  generating 
options  and  predicting  (i.e.,  labelled  "estimate"  in  the 
categorization) .  Only  three  of  the  seven  monitoring  measures  met 
the  inclusion  criteria.  Of  the  remaining  20  ACCES  measures  that 
deal  with  internal  staff  and  command  post  interactions  and 
compatability  only  two  met  the  criteria  for  inclusion.  Therefore 
we  were  able  to  use  fewer  than  half  of  the  ACCES  measures  and  we 
have  little  indication  of  how  coordination  within  the  division 
staff  might  have  affected  performance  on  the  other  measures. 

The  following  are  the  definitions  of  the  measures  we  selected, 
listed  under  their  respective  categories. 


CATEGORY  TITLE 

OVERALL  PLAN 

EFFECTIVENESS  QUALITY 


PLAN 

CONGRUENCE 


DEFINITION 


Number  of  plan  assignments  that 
remain  in  force  unchanged  for  the 
intended  period,  expressed  as  a 
percentage  of  the  total  assign¬ 
ments.  The  assignments  in  a  plan 
are:  mission,  assets,  boundaries, 

and  schedules.  The  intended  peri¬ 
od  is  the  time  stated  in  the  im¬ 
plementing  directive  or  command 
briefing.  The  time  the  plan  is  in 
force  is  the  time  from  implementa¬ 
tion  to  the  time(s)  the  plan  is 
changed  or  abandoned. 

A  minor  change  (incongruence) 
to  a  plan  is  one  that  requires  no 
change  to  the  basic  plan  and  in¬ 
cludes  such  things  as  adjustments 
to  fires.  A  moderate  change  is 
the  implementation  of  existing 
alternatives  and  generally  in¬ 
cludes  the  execution  of  on-order 
missions.  Major  changes  are  those 
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involving  changes  in  one  or  more 
of  the  four  types  of  assignments 
(see  PLAN  QUALITY)  and  for  which 
no  previous  planning  has  been 
done.  Plan  Congruence  is  the 
number  of  control  cycles  arising 
from  minor  incongruences  plus  .8 
times  the  number  of  control  cycles 
arising  from  moderate  incongru¬ 
ences,  expressed  as  a  percentage 
of  the  total  number  of  control 
cycles . 

PLAN  CYCLE  Median  time  used  to  complete  the 

TIME  control  cycle.  The  time  is  meas¬ 

ured  from  the  headquarters  percep¬ 
tion  of  an  event  until  a  directive 
is  issued  or  a  decision  is  made  to 
do  nothing. 

PLAN  LEAD  Number  of  directives  for  which 

TIME  ADEQUACY  planning  lead  time  provided  to 

subordinates  is  adequate,  express¬ 
ed  as  a  percentage  of  the  total 
number  of  directives.  Adequacy  of 
lead  time  is  defined  here  as 
meeting  the  generally  accepted  1/3 
-  2/3  rule  in  which  the  command 
uses  no  more  than  1/3  of  the  time 
remaining  before  implementation  to 
issue  its  order  to  subordinates. 

MONITORING  ACCURACY  Number  of  units  for  which  head¬ 

quarters  data  are  within  the  de¬ 
sired  window,  expressed  as  a  per¬ 
centage  of  the  total  number  of 
units  monitored.  Accuracy  is 
reported  for  enemy  and  own  units 
separately.  The  desired  window 
is  here  defined  as  within  a  radius 
of  one  kilometer  from  actual 
location  for  own  units  and  within 
two  kilometers  for  enemy  units. 

IMPACT  ON  PLAN  Number  of  control  cycles  initiated 

because  of  monitoring  errors, 
expressed  a^  the  percentage  of 
total  control  cycles  initiated, 
subtracted  from  100  percent. 

UNDERSTANDING  COMPLETENESS  Percentage  of  understandings  of 

the  enemy  situation  presented  at 
formal  briefings  which  are  com¬ 
plete  understandings  (i.e.,  in- 
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elude  composition,  disposition, 
courses  of  action,  combat  power, 
and  activity) . 


ESTIMATE 


QUALITY  Number  of  understandings,  of  the 

situation  held  by  the  headquar¬ 
ters  which  were  correct,  divided 
by  the  total  number  of  under¬ 
standings.  Quality  scores  are 
reported  separately  for  enemy  and 
own  understandings. 


IMPACT  ON  PLAN  Number  of  control  cycles  caused 

by  headquarter's  misunderstand¬ 
ing  of  the  situation,  expressed  as 
a  percentage  of  total  control 
cycles  initiated  and  subtracted 
from  100  percent. 

UNDERSTANDING  Median  time  from  the  expression  of 
TIME  an  understanding  to  the  end  of  the 

period  which  the  understanding 
covers. 


MULTIPLE 

PLANNERS 


MULTIPLE 

OPTIONS 


PREDICTION 

COMPLETENESS 


PREDICTION 

QUALITY 


Number  of  control  cycles  for  which 
two  or  more  staff  members  partici¬ 
pated  in  the  development  of 
course (s)  of  action,  expressed  as 
a  percentage  of  the  total  number  of 
control  cycles. 

Number  of  control  cycles  for  which 
more  than  one  possible  course  of 
action  was  considered,  expressed 
as  a  percentage  of  the  total  num¬ 
ber  of  control  cycles. 

Number  of  courses  of  action  pre¬ 
sented  at  formal  briefings  that 
include  predictions  of  enemy 
reaction,  degree  of  mission  ac¬ 
complishment,  and  residual  capac¬ 
ity  of  friendly  and  enemy  units 
involved,  expressed  as  a  percent¬ 
age  of  *'he  total  number  of  courses 
of  action  presented. 

Number  of  predictions  which 
turned  out  to  be  correct,  divided 
by  the  total  number  of  predic¬ 
tions  recorded. 
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Number  of  assignments  in  imple¬ 
menting  directives  which  do  not 
contradict  the  assignment  of  the 
Commander's  decision(s), 
expressed  as  a  percentage  of  the 
total  number  of  assignments  in  the 
directives . 

DIRECTIVES  NOT  Number  of  directives  not  queried 
QUERIED  by  recipients,  expressed  as  a  per¬ 

centage  of  the  total  number  of 
directives . 

Number  of  information  reports  not 
queried  by  recipients,  expressed 
as  a  percentage  of  the  total  num¬ 
ber  of  information  reports. 

COORDINATION  UNDERSTANDING  Degree  of  agreement  between  cells 

COMPARABILITY  in  a  node,  concerning  understand¬ 
ings  of  the  comparable  friendly 
and  enemy  situations,  expressed  as 
a  percentage  of  all  such  under¬ 
stand!  3  in  the  network. 

Lessons  Learned 

This  section  will  discuss  lessons  learned  in  the  two  areas  of 
concern  of  this  study,  performance  of  the  units  observed  and  the 
psychometric  strengths  and  weaknesses  of  the  ACCES  methodology. 

Unit  Performance 

ACCES  Scores.  Table  1  presents  the  scores  on  20  ACCES  meas¬ 
ures  for  the  six  divisions  which  compose  the  sample  for  this 
study.  The  20  measures  presented  are  ones  for  which  there  was 
approximately  uniform  data  collection  across  all  units  examined. 
The  table  also  presents  ranges  and  means  of  the  measures  for  the 
divisions.  In  order  to  conceal  the  identity  of  specific  divi¬ 
sions,  they  have  been  randomly  assigned  the  letters  A-F  for  pres¬ 
entation  here. 

Of  the  percentage  (as  opposed  to  time)  scores  examined,  those 
which  have  the  highest  average  score  are  Plan  Lead  Time  Adequacy 
(M=92) ,  Monitoring  Impact  on  Plan  (M=96) ,  and  Plan  Consistency 
(M=98) .  Those  which  have  the  lowest  average  scores  are  Monitor¬ 
ing  Accuracy  -  Enemy  (M=55) ,  Monitoring  Accuracy  -  Own  (M=50) , 
Understanding  Completeness  -  Enemy  (M=52) ,  and  Prediction 
Completeness  (M=54). 

The  percentage  scores  which  have  the  most  variability  are  Plan 
Quality  (R=58) ,  Monitoring  Accuracy  -  Enemy  (R=51) ,  Monitoring 
Accuracy  -  Own  (R=32) ,  Understanding  Impact  on  Plan  (R=31) , 
Multiple  Options  (R=39) ,  and  Prediction  Quality  (R=34) .  Although 


REPORTS  INFORMATION 

NOT  QUERIED 


PLAN 

CONSISTENCY 


7 


Table  1 
ACCES  SCORES 


ACCES  PERCENTAGE  SCORES 

A- 

~E 

"T 

1.  Plan  Quality 

98 

55 

71 

40 

71 

95 

72 

58 

2.  Plan  Congruence 

90 

68 

72 

72 

74 

68 

74 

22 

3.  Plan  Lead  Time  Adequacy 

93 

80 

95 

97 

89 

96 

92 

17 

4.  Monitoring  Accuracy  (Enemy) 

51 

26 

77 

74 

54 

50 

55 

51 

5.  Monitoring  Accuracy  (Own) 

37 

40 

44 

53 

69 

57 

50 

32 

6.  Monitoring  Impact  on  Plan 

95 

100 

97 

94 

97 

95 

96 

06 

7.  Understanding  Impact  on  Plan 

98 

67 

86 

80 

90 

79 

83 

31 

8.  Understanding  Quality  (Enemy) 

87 

78 

73 

76 

72 

— 

77 

15 

9.  Understanding  Quality  (Own) 

— 

73 

77 

80 

73 

— 

76 

07 

10.  Understanding  Completeness 

(Enemy) 

— 

65 

28 

55 

62 

— 

52 

37 

11.  Directive  Not  Queried 

85 

93 

89 

86 

— 

89 

88 

08 

12 .  Plan  Consistency 

100 

98 

98 

96 

— 

97 

98 

04 

13.  Multiple  Planners 

96 

83 

88 

84 

83 

93 

88 

13 

14.  Multiple  Options 

53 

68 

92 

56 

80 

81 

72 

39 

15.  Prediction  Completeness 

— 

69 

39 

40 

68 

— 

54 

30 

16.  Prediction  Quality 

81 

88 

64 

79 

54 

84 

75 

34 

17.  Information  Not  Queried 

— 

81 

86 

83 

80 

— 

82 

06 

18.  Understanding  Comparability 

81 

85 

95 

75 

75 

80 

82 

20 

MEAN= 

82 

73 

76 

73 

74 

77 

76 

24 

ACCES  TIME  SCORES 

19.  Plan  Cycle  Time 

31 

20 

47 

42 

44 

35 

36 

27 

(median,  min.) 

20.  Understanding  Time 

— 

00 

24 

09 

04 

— 

12 

24 

(median,  hrs.) 
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Understanding  Completeness-Enemy  has  a  range  37,  it  is  not  in¬ 
cluded  as  a  measure  with  high  variability  because  data  were  ob¬ 
tained  from  only  four  of  the  divisions  in  the  sample.  Those 
measures  with  the  least  variability  are  Monitoring  Impact  on  Plan 
(R=6)  ,  Understanding  Quality  -  Own  (R=7),  Directives  Not  Queried 
(R— 8 ) ,  Plan  Consistency  (R=4) ,  and  Information  Not  Queried  (R=6) . 

The  two  time  scores.  Plan  Cycle  time  and  Understanding  Time, 
are  displayed  at  the  bottom  of  Table  1.  The  mean  Plan  Cycle  Time 
for  divisions  is  36  minutes.  This  is  the  mean  of  the  six  divi¬ 
sion  times,  each  of  which  is  a  median  of  the  Plan  Cycle  Time 
score  for  that  division.  This  indicates  that,  on  the  average,  36 
minutes  elapsed  between  the  time  the  divisions  perceived  the  need 
for  a  decision  and  the  actual  dissemination  of  that  decision. 
Understanding  Time  scores  are  expressed  in  median  hours.  On  the 
average,  division  Understanding  Time  was  13  hours.  However,  one 
division  scored  zero  (i.e.  less  than  30  minutes)  on  the  Under¬ 
standing  Time  measure  which  indicates  that  the  majority  of  the 
understandings  expressed  by  this  unit  concerned  the  immediate 
tactical  situation.  No  data  are  available  for  two  divisions  on 
this  measure. 

Rank  orderings.  Six  percentage  scores  which  had  ranges  great¬ 
er  than  30  were  selected  for  further  study  of  the  rank  order  of 
the  divisions  on  these  scores  plus  Plan  Cycle  Time,  the  only  time 
measure  for  which  there  was  a  datum  for  each  division.  The  vari¬ 
ables  with  the  highest  ranges  were  chosen  in  order  that  the  rank 
orderings  would  be  relatively  clear  and  unambiguous.  These 
scores  were  extracted  from  Table  1  and  are  reproduced  in  Table  2. 
The  ranks  of  the  divisions  on  each  variable  are  given  in  Table  3 
with  the  mean  rank  and  the  range  of  the  ranks  given  at  the  bottom 
of  the  table.  Note  that  for  the  percentage  scores  the  highest 
score  is  given  the  rank  of  "1"  while  for  the  time  scores  the  rank 
of  "1"  is  given  to  the  lowest,  or  fastest. 

The  mean  division  rankings  all  fall  in  the  interval  of  3-4  so 
it  would  appear  that  none  was  outstanding  in  either  direction. 

It  is  interesting  to  note  that  the  one  (F)  with  a  mean  of  "3"  had 
no  rank  of  either  "1"  or  "6"  and  thus  also  had  the  smallest 
range.  Even  though  it  was  not  preeminent  on  any  of  these  meas¬ 
ures  it  still  attained  the  highest  mean  rank  by  virtue  of  never 
being  worst. 

The  scores  of  Table  2  were  correlated  using  Pearson's  method 
and  the  ranks  of  Table  3  were  corelated  using  Spearman's  method. 
These  correlations  are  given  in  Table  4  with  the  Spearman  coeffi¬ 
cients  below  the  major  diagonal  and  the  Pearson  coefficients 
above.  With  only  six  observations,  the  standard  error  of  the 
correlation  coefficient  is  greater  than  .4,  so,  very  roughly 
speaking,  the  numbers  in  Table  4  may  be  considered  known  within  a 
factor  of  plus  or  minus  .8.  Thus,  we  can,  if  we're  willing  to  go 
out  on  a  bit  of  a  limb,  say  that  there  may  be  a  negative  correla¬ 
tion  between  Monitoring  Accuracy  (Enemy)  on  the  one  hand  and  Pre¬ 
diction  Quality  and  Plan  Cycle  Time  on  the  other.  Prediction 
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Table  2 


ACCES  Scores  with  greatest  variability 


ACCES  PERCENTAGE  SCORES 

A 

Division 

BCD 

E 

F 

MEAN 

RANGE 

■E3B 

(R) 

1. 

Plan  Quality 

98 

55 

71 

40 

71 

95 

72 

58 

4  . 

Monitoring  Accuracy  (Enemy) 

51 

26 

77 

74 

54 

50 

55 

51 

5. 

Monitoring  Accuracy  (Own) 

37 

40 

44 

53 

69 

57 

50 

32 

7. 

Understanding  Impact  on  Plan 

98 

67 

86 

80 

90 

79 

83 

31 

14  . 

Multiple  Options 

53 

68 

92 

56 

80 

81 

72 

39 

16. 

Prediction  Quality 

81 

88 

64 

79 

54 

84 

75 

34 

ACCES  TIME  SCORES 

19. 

Plan  Cycle  Time 

31 

20 

47 

42 

44 

35 

36 

27 

(median,  min.J 

Table  3 

Rank  order  of  divisions  on  measures  with 

greatest 

:  variability 

Division 

ACCES  Variable 

A 

B 

C 

D 

E 

F 

1. 

Plan  Quality 

1 

5 

3 . 5 

6 

3 . 5 

2 

4  . 

Monitoring  Accuracy  (Enemy) 

4 

6 

1 

2 

3 

5 

5. 

Monitoring  Accuracy  (Own) 

6 

5 

4 

3 

1 

2 

7. 

Understanding  Impact  on  Plan  1 

6 

3 

4 

2 

5 

14. 

Multiple  Options 

6 

4 

1 

5 

3 

2 

16. 

Prediction  Quality 

3 

1 

5 

4 

6 

2 

19. 

Plan  Cycle  Time 

2 

1 

6 

4 

5 

3 

(median,  min. ) 

MEAN= 

3  . 

3 

4 

3.4 

4 

3.4 

3 

RANGE= 

6 

6 

6 

5 

6 

4 
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Table  4 


Spearman\Pearson  correlations  between  selected  ACCES  variables 


ACCES  Variable 

ACCES 

Variable 

1. 

4. 

5. 

7  . 

14  . 

16. 

19. 

1. 

Pln.Qual . 

1.00 

-.17 

-.09 

.55 

.  15 

.  03 

.  08 

4 . 

M. A. (Enm. ) 

-.19 

1.00 

.  19 

.43 

.  19 

-.48 

-.90 

5. 

M . A . (Own) 

-.16 

.20 

1.00 

-.07 

.  36 

-.59 

-.53 

7  . 

UIoP 

.  53 

.49 

-.09 

1.00 

-.11 

-.50 

-.50 

14  . 

Mult.  Opt. 

.04 

.26 

.49 

-.26 

1.00 

-.53 

-.44 

16. 

Pred.  Qual. 

.07 

-.83 

-.49 

-.66 

-.26 

1.00 

.78 

19. 

PC  Time 

.  10 

-.89 

-.54 

-.37 

-.60 

.  89 

1.00 

Quality  and  Plan  Cycle  Time  may  be  positively  correlated. 

It  is  important  to  recognize  that  no  independent  criterion  ex¬ 
ists  against  which  to  validate  the  ACCES  scores.  Without  such 
criteria,  it  would  be  a  mistake  to  judge  that  those  divisions  ob¬ 
taining  the  highest  overall  ACCES  scores  were  those  which  truly 
performed  best.  As  a  matter  of  fact,  one  division  which  was  gen¬ 
eral1  y  judged  at  the  time  of  the  exercise  to  have  performed  in  a 
superior  manner  does  not  have  the  highest  overall  ACCES  score, 
nor  is  it  highest  in  three  of  the  primary  measures  (Plan  Quality, 
Plan  Congruence,  Plan  Lead  Time  Adequacy) .  It  is,  however,  high¬ 
est  in  Plan  Cycle  Time,  which  on  the  face  of  it,  would  appear  to 
be  the  poorest  performance  on  this  measure.  This  same  division 
was  highest  on  Monitoring  Accuracy- (Enemy) ,  Multiple  Options,  In¬ 
formation  Not  Queried,  and  Understanding  Comparability,  and  it 
was  the  lowest  of  all  divisions  on  Understanding  Completeness- 
(Enemy)  and  on  Prediction  Completeness. 

Of  the  divisions  observed,  no  one  division  clearly  scored 
higher  than  the  others  on  the  majority  of  the  ACCES  measures. 
There  were,  in  contrast,  many  measures  on  which  there  seemed  to 
be  relatively  little  variability  between  the  divisions.  As  a 
matter  of  fact,  on  half  of  the  measures,  the  highest  and  lowest 
scoring  divisions  were  less  than  29  percentage  points  apart.  On 
no  measure  did  any  two  divisions  differ  by  as  much  as  60  per¬ 
centage  points. 

All  divisions  observed  had  high  scores  on  the  measure  of  Plan 
Consistency  (R=100-96) .  This  measure  assessed  the  degree  to 
which  the  Commander’s  guidance  was  incorporated  into  the  opera¬ 
tions  plan.  Apparently  units  had  no  difficulty  in  understanding 
and  implementing  their  Commander's  intent. 
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Scores  were  generally  high  on  the  measure  of  Plan  Lead  Time 
Adequacy,  which  assessed  the  frequency  with  which  units  were  able 
to  adhere  to  the  doctrinal  1/3-2/3  rule  for  order  dissemination. 
According  to  this  measure,  units  were  successful  in  satisfying 
this  principle  in  80-97%  of  the  decision  situations. 

The  divisions  observed  also  scored  well  on  the  measure  of  Mon¬ 
itoring  Impact  on  Plan.  This  measure  assessed  the  impact  of  a 
unit's  monitoring  accuracy  on  plan  execution.  The  results  indi¬ 
cate  that  plans  were  generally  unaffected  by  the  units'  monitor¬ 
ing  accuracy,  i.e.  plans  seldom  had  to  be  changed  due  to  monitor¬ 
ing  errors. 

Also,  results  show  that  the  divisions  observed  usually  involv¬ 
ed  more  than  one  planner  in  the  planning  process  (88%  of  the 
time,  on  the  average)  and  that  12%  (on  the  average)  of  the  direc¬ 
tives  issued  were  queried  by  the  recipients. 

In  summary  then,  the  divisions  observed  demonstrated  strengths 
in  their  ability  to  involve  multiple  staff  members  in  the  formu¬ 
lation  of  clear  plans  in  accordance  with  their  commander's  in¬ 
tent,  and  to  issue  them  in  a  timely  manner.  Indicative  of  the 
difficulty  in  inferring  high  or  low  performance  from  these  scores 
is  the  apparently  small  effect  of  monitoring  errors  on  planning. 
This  would  appear  to  indicate  little  error  in  monitoring.  How¬ 
ever,  that  conclusion  is  unjustified  as  can  be  seen  from  low 
scores  on  the  specific  measures  of  Monitoring  Accuracy  -  both 
(Own)  and  (Enemy) .  These  scores,  in  turn,  may  not  be  what  they 
seem,  as  will  be  discussed  in  a  later  section  of  this  paper. 

Other  measures  on  which  the  divisions  scored  relatively  low 
were  two  measures  which  assessed  briefing  quality  -  Understanding 
Completeness  and  Prediction  Completeness.  This  means  that  the 
understanding  and  predictions  were  not  completely  or  fully 
briefed. 

ACCES  Measures 


This  discussion  will  be  limited  to  the  twenty  measures  that 
appear  in  Table  1. 

Measures  of  overall  effectiveness.  Cycle  Time  Adequacy  is  the 
only  primary  measure  for  which  there  were  insufficient  data  for 
inclusion  in  this  discussion.  According  to  Adaptive  Control 
System  Theory  upon  which  ACCES  is  built,  one  would  expect  posi¬ 
tive  correlations  between  Plan  Quality,  Plan  Congruence,  and  Plan 
Lead  Time  Adequacy.  Negative  correlations  would  be  expected  be¬ 
tween  these  variables  and  Plan  Cycle  Time.  In  other  words,  a  di¬ 
vision  which  is  doing  well  should  have  consistently  high  scores 
on  the  three  percentage  based  scores  and  low  (fast)  Plan  Cycle 
Times . 
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One  of  the  difficulties  with  relying  almost  exclusively  on 
percentage  measures  in  this  context  is  that  the  frequency  basis 
for  a  measure  tends  to  vary  considerably.  On  one  occasion  a 
score  of  75%  for  Plan  Quality  may  be  based  on  30  unchanged 
assignments  out  of  a  total  of  40  while  on  another  the  same  score 
could  represent  300  out  of  400.  This  problem  is  compounded  by 
the  fact  that  the  overall  score  will  always  be  less  than  any  of 
the  daily  scores  because  of  the  fact  that  assignments  can  carry¬ 
over  beyond  a  single  day.  Thus,  for  example,  50  assignments  on 
one  day  and  60  assignments  on  the  following  day  may  represent 
anywhere  from  60  (if  all  assignments  carried  over  from  the 
previous  day)  to  110  (if  none  did)  total  assignments. 

One  Plan  Congruence  problem  is  that  it  is  not  possible  from 
the  percentage  score  to  know  the  balance  among  minor,  moderate, 
and  major  incongruences.  The  purpose  of  this  measure  is  to  indi¬ 
cate  the  average  severity  of  changes  made  to  directives.  By  cal¬ 
culating  the  numerator  as  "number  of  minor  changes  plus  .8  times 
the  number  of  moderate  changes,"  the  absolute  number  of  major 
changes  was  lost  (until  the  last  two  exercises)  and  the  division 
which  issued  few  directives  tended  to  be  penalized  compared  with 
a  division  that  had  to  issue  many  more  directives  but  had  the 
same  number,  or  even  more,  of  major  change  directives. 

Perhaps  an  example  will  help  to  clarify  the  issues.  Suppose 
we  have  a  three  phase  mission.  One  division  issues  only  the  pri¬ 
mary  directives  for  each  phase  plus  one  change  directive  within 
each  phase,  of  which  two  are  major  changes.  Their  Plan  Congru¬ 
ence  score  is  4/6  or  67%.  A  second  division  with  the  same  three 
phase  mission  issues  five  change  directives  per  phase  of  which 
three  are  major  changes,  three  moderate,  and  nine  minor.  Their 
Plan  Congruence  score  is  14.4/18  or  80%,  considerably  higher  than 
the  other  division  even  though  they  had  to  issue  many  more  change 
directives  including  more  moderate  and  more  major  ones.  It  would 
appear  that  allowing  the  numerator  to  grow  with  the  frequency  of 
minor  changes  tends  to  obscure  the  effect  intended  to  be  meas¬ 
ured.  In  conjunction  with  total  number  of  changes  this  measure 
of  "average  severity  of  change"  is  more  meaningful.  One  is  less 
likely  to  be  misled  by  Plan  Congruence  of  67%  and  80%  when  the 
total  numbers  of  cycles  is  known  (6  and  18) . 

We  have  been  referring  to  these  measures  as  "percentage" 
scores  but  one  usually  thinks  of  percentages  as  numbers  which 
range  from  zero  to  100.  That  is  not  the  case  for  Plan  Quality 
and  Plan  Congruence.  Having  no  major  or  moderate  changes  would 
result  in  scores  of  100%,  but  because  of  the  way  the  measures  are 
defined  it  is  impossible  to  achieve  a  score  of  zero%.  There  must 
be  at  least  four  more  total  assignments  than  there  are  changes 
in  assignments,  and  the  total  control  cycles  must  be  at  least  one 
more  than  the  number  of  control  cycles  arising  from  minor,  moder¬ 
ate,  or  major  incongruence. 
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The  Plan  Quality  score  does  have  a  unique  meaning  when  consid¬ 
ered  in  relation  to  Plan  Congruence:  Plan  Quality  indicates  the 
severity  of  major  changes  only.  For  example,  division  C  and  D 
(Table  1)  had  the  same  Plan  Congruence  score  (72)  but  differed 
considerably  in  their  Plan  Quality  scores  (71  vs  40) .  This  indi¬ 
cates  that,  although  the  two  divisions  had  the  same  average 
severity  of  change,  division  D  had  more  assignment  changes  per 
major  change  than  did  division  C. 

The  apparent  purpose  of  Plan  Cycle  Time  is  to  indicate  how 
fast  the  command  and  control  system  was  reacting.  The  distribu¬ 
tion  of  cycle  times  is  highly  skewed  since  a  large  number  of 
"minor"  decisions  are  made  instantaneously  while  a  few  "major" 
decisions  require  considerable  analysis  and  coordination.  The 
median  of  this  distribution  was  chosen  as  Plan  Cycle  Time.  How¬ 
ever,  a  danger  is  that  even  most  median  values  will  equal  zero 
because  of  the  large  number  of  zeros  in  the  distribution.  This, 
of  course,  then  allows  no  meaningful  comparisons  between  divi¬ 
sions,  command  posts,  or  between  times  at  the  same  command  post. 

A  possible  remedy  for  this  problem  would  be  to  report  separately 
the  number  of  zeros  and  then  also  to  report  the  number  and  median 
of  all  the  remaining  (non-zero)  cycle  times. 

If  we  make  the  assumption  that  it  takes  longer  to  make  major 
changes  than  minor  ones,  then  Plan  Cycle  Time  should  correlate 
negatively  with  both  Plan  Quality  and  Plan  Congruence.  Measure¬ 
ment  eccentricities  that  could  drive  down  these  correlations  in¬ 
clude  the  effects  of  a  high  number  of  "moderate"  changes  on  Plan 
Congruence  scores  —  moderate  changes  are  typically  the  execution 
of  on-order  missions  that  can  be  implemented  very  rapidly,  per¬ 
haps  faster  than  many  minor  changes  which  might  require  more 
coordination.  Defining  the  communication  of  warning  orders  as 
ending  a  cycle  could  drive  down  Plan  Cycle  Time  for  "major" 
changes  as  near  immediate  sending  of  warning  orders  to  subordi¬ 
nates  upon  receipt  of  a  directive  from  higher  HQ  is  SOP  with  many 
commands  even  though  the  final  decision  may  require  much  further 
analysis  and  debate.  This  could  reduce  the  negative  correlation 
with  both  Plan  Quality  and  Plan  Congruence. 

Plan  Lead  Time  Adequacy  is  a  criterion-based  measure  which  is 
similar  to  Plan  Cycle  Time  and  helps  to  interpret  Plan  Cycle  Time 
in  terms  of  its  operational  effect.  If  we  again  assume  that  ma¬ 
jor  changes  should  require  more  planning  time  than  minor  ones, 
then  Plan  Lead  Time  Adequacy  should  correlate  positively  with 
Plan  Quality  and  Plan  Congruence.  It  should  correlate  negatively 
with  Plan  Cycle  Time.  This  should  be  the  most  direct  and  criti¬ 
cal  measure  of  how  proactive  the  command  is.  If  they  are  looking 
ahead  and  planning  accordingly,  if  they  are  "in  control  of  the 
situation"  and  not  merely  reacting  to  what  the  enemy  has  done, 
this  score  should  be  high.  For  a  truly  proactive  command,  it 
should  be  independent  of  the  other  three  measures  but  correlate 
positively  with  Understanding  Time  and  Prediction  Time,  at  least. 
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Monitoring  measures.  Of  the  seven  monitoring  measures,  three 
will  be  addressed  in  this  section:  Monitoring  Accuracy  (Enemy) , 
Monitoring  Accuracy  (Own) ,  and  Monitoring  Impact  on  Plan.  The 
two  accuracy  measures  are  among  the  four  having  the  lowest  per¬ 
centages  of  the  eighteen  being  considered.  Monitoring  Impact  on 
Plan,  on  the  other  hand,  is  second  highest  of  all  these  measures. 
This  would  appear  to  be  a  discrepancy  since  one  might  expect  in¬ 
accuracies  of  monitoring  to  require  initiation  of  unanticipated 
planning  cycles.  However,  this  seeming  discrepancy  may  be  due  to 
too  stringent  criteria  for  monitoring  accuracy.  In  all  of  the 
exercises  thus  far  a  criterion  of  one  kilometer  accuracy  for 
friendly  units  and  two  kilometers  accuracy  for  enemy  units  has 
been  applied.  In  view  of  the  fact  that  a  U.S.  battalion  task 
force  defending  in  sector  covers  an  area  of  between  32  and  80 
square  kilometers  (FM  71-2) ,  the  criterion  of  one  kilometer  accu¬ 
racy  for  locating  its  center  of  mass  may  be  unrealistic.  Given 
the  greater  uncertainty  regarding  the  exact  location  of  enemy 
units,  requiring  accuracy  within  two  kilometers  for  locating 
their  center  of  mass  is  even  more  unrealistic. 

Understanding.  All  the  understanding  measures  had  sufficient 
data  (at  least  four  scores)  to  be  included  in  this  report: 
Understanding  Completeness,  Understanding  Quality  (Enemy) ,  Under¬ 
standing  Quality  (Own) ,  Understanding  Impact  on  Plan,  and  Under¬ 
standing  Time;  but  it  may  be  noted  from  Table  1  that  several  of 
these  data  points  are  missing. 

Prediction  Completeness  will  be  discussed  in  a  later  section, 
but  it  and  Understanding  Completeness  had  among  the  lowest  scores 
of  all  the  measures.  This  may  be  because  of  the  stringent  re¬ 
quirement  that  to  be  "complete"  an  Understanding  must  include  all 
five  of  the  following:  composition,  disposition,  courses  of 
action,  combat  power,  and  activity,  and  a  Prediction  must  in¬ 
clude:  predictions  of  enemy  reaction,  degree  of  mission  accom¬ 
plishment,  and  residual  capacity  of  friendly  and  enemy  units  in¬ 
volved.  The  assumption  that  all  of  these  would  be  required  every 
time  an  understanding  or  prediction  is  briefed  regardless  of 
their  roles  in  previous  briefings  is  somewhat  ritualistic  and  may 
not  be  yielding  the  most  informative  measures. 

Understanding  Completeness  cannot  be  directly  compared  with 
Understanding  Quality  (both  Enemy  and  Own)  because  the  latter  in¬ 
cludes  all  understandings  whereas  the  former  includes  only  those 
expressed  in  formal  briefings.  It  might  prove  useful  to  separate 
the  count  for  Understanding  Quality  into  two  parts:  those 
perceptions  expressed  in  formal  briefings  and  the  remainder. 

Doing  this  would  yield  the  same  measures  as  now  and,  in  addition, 
would  give  a  measure  of  quality  for  formal  briefings. 
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Future  Work 


Data  Base 


As  of  now,  the  ACCES  data  base  consists  of  the  percentages  and 
times  given  in  Table  1.  The  percentages  are  the  results  of  wide¬ 
ly  varying  numerators  and  denominators  only  a  few  of  which  are 
now  available.  Many  of  the  original  logs  of  the  observers  are  on 
hand  but  the  possibility  of  reconstructing  scores  from  these  is 
questionable.  A  search  is  being  conducted  for  the  intermediate 
data  sheets  from  which  counts  will  be  reconstructed.  At  that 
time  the  raw  data  will  be  entered  into  a  file  which  will  permit 
manipulations  not  now  possible.  These  data  will  still  be  too 
sparse  to  permit  clear  statistical  inferences  but  it  will  be 
possible  to  search  for  relationships  between  more  basic, 
elemental  variables  than  the  percentages  now  available. 

Eventually,  it  will  be  necessary  to  have  the  data  base  com¬ 
prise  several  files,  but  for  the  data  expected  to  be  available  in 
the  immediate  future,  one  should  suffice.  It  will  consist  of  the 
data  which  were  used  to  calculate  the  percentages  in  Table  1,  and 
include  things  like  (for  a  specified  period)  Number  of  Control 
Cycles,  Total  Number  of  Assignments,  Number  of  Assignment 
Changes,  Number  of  Major  Incongruences,  etc.  Figure  2  gives  an 
approximation  of  what  a  record  in  this  file  would  contain:  on 
the  left  are  given  the  names  of  the  variables  and  on  the  right, 
for  each  variable  is  the  length  of  the  field  (number  of  columns) 
necessary  for  that  variable.  Much  of  the  detail  indicated  for 
this  file  will  be  unavailable  for  past  exercises  but  all  the 
variables  for  which  there  are  data  will  be  included.  In 
addition,  summaries  of  some  of  the  variables  listed  for  files  2-5 
(see  Figure  3)  will  be  included,  i.e.,  some  of  this  information 
is  expected  to  be  available,  not  in  the  detailed  form  indicated 
here  but  as  they  have  been  accumulated  in  the  past.  These  file 
definitions  are  intended  to  be  more  prospective  than  retrospec¬ 
tive  in  that  they  define  variables  which  will  be  available  in 
future  exercises  but  are  not,  for  the  most  part,  recoverable 
from  past  exercises. 

Those  variables  in  File  1  under  Monitoring  which  have  to  do 
with  Monitoring  Accuracy  have  posed  a  potential  security  problem. 
The  problem  has  to  do  with  recording  map  coordinates  of  unit 
positions.  The  way  in  which  the  problem  has  been  avoided  in  the 
past  has  been  to  record  only  whether  or  not  a  unit  was  plotted 
within  a  given  standard  of  accuracy,  one  grid  square  for  own  and 
two  grid  squares  for  enemy  maneuver  battalions.  Relatively  poor 
performance  on  these  variables  may  have  been  due  to  an  overly 
stringent  criterion.  If  it  is  possible  to  record  amount  of  dis¬ 
crepancy  between  the  plotted  location  of  a  maneuver  battalion  and 
its  ground  truth  location,  that  should  prove  to  be  a  better  meas¬ 
ure  than  recording  whether  or  not  some  arbitrary  criterion  was 
attained.  The  security  problem  would  still  be  avoided  by 
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Variable  Field  Length 

FILE  1: 

1.  Day  (or  12  or  6  hr.  period)  of  exercise 

2.  Unit  (A,  B,  C,  etc.) 

Overall  Effectiveness 

3.  #  decision  cycles  started 

4.  #  decision  cycles  completed 

5.  #  decision  cycles  interrupted  by  anticipated  change 

6.  #  decision  cycles  interrupted  by  major  change 

i.e.  change  in  mission,  assets,  boundary, 
and  schedule  (MABS) 

7.  #  assignments  made  (MABS),  including  changed 

8.  #  assignments  changed  (MABS) 

9.  #  assignments  which  contradict  Cmdr's  decision 

10.  |  directives  issued 

11.  #  directives  queried 

Monitoring 

12.  #  Own  Force  maneuver  battalions  (MBs) 

13.  |  Own  Force  maneuver  battalions  plotted  in  DMAIN  INTEL 

14.  #  OF  MBs  plotted  within  one  grid  square  of  correct 

15.  *  Own  Force  maneuver  battalions  plotted  in  DMAIN  OPS 

16.  #  OF  MBs  plotted  within  one  grid  square  of  correct 

17.  #  OF  MBs  plotted  on  DTAC  INTEL  maps 

18.  #  OF  MBs  plotted  within  one  grid  square  of  correct 

19.  f  OF  MBs  plotted  on  DTAC  OPS  maps 

20.  #  OF  MBs  plotted  within  one  grid  square  of  correct 

21.  #  Enemy  maneuver  battalions  (MBs) 

22.  #  Enemy  MBs  plotted  on  DMAIN  INTEL  maps 

23.  #  Enemy  MBs  plotted  within  two  grid  squares  of  correct 

24.  #  Enemy  MBs  plotted  on  DMAIN  OPS  maps 

25.  #  Enemy  MBs  plotted  within  two  grid  squares  of  correct 

26.  #  Enemy  MBs  plotted  on  DTAC  INTEL  maps 

27.  #  Enemy  MBs  plotted  within  two  grid  squares  of  correct 

28.  #  Enemy  MBs  plotted  on  DTAC  OPS  maps 

29.  #  Enemy  MBs  plotted  within  two  grid  squares  of  correct 

30.  #  Friendly  units  of  interest 

31.  #  Friendly  Units  w/data  outside  desired  time  window 

32.  #  Queries  re  above  friendly  units 

33.  #  Enemy  units  of  interest 

34.  #  Enemy  Units  w/data  outside  desired  time  window 

35.  #  Queries  re  above  enemy  units 

36.  #  Decision  cycles  initiated  because  of  monitoring 

errors 

37.  #  Predictions  of  time  at  which  weather  will  change 

38.  #  Correct  weather  change  predictions 

Understandings 

39.  #  Understandings  of  enemy  situation  presented  at 

formal  briefings 
#  Understandings  which  include: 

40.  a.  composition 

41.  b.  disposition 

42.  c.  courses  of  action 

43.  d.  combat  power 

44.  e.  activity 

45.  #  Understandings 

46.  #  Perceptions  of 

47.  #  Perceptions  co _ 

48.  #  Perceptions  not  incorrect 

49.  #  Perceptions  incorrect 

50.  #  Predictions  correct 

51.  |  Predictions  not  incorrect 

52.  #  Predictions  incorrect 

53.  #  Decision  cycles  initiated  because  of 

errors  of  understanding  2 

Figure  2.  ACCES  data  file. 
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Variable 


* 


1. 

2. 

3  . 

4  . 

5. 

6 . 

7. 

8. 


1. 

2  . 

3  . 

4  . 
5. 


1. 

2  . 

3  . 

4  . 

5. 

6. 

7. 

8. 

9. 

10. 


1. 

2  . 

3  . 

4  . 

5. 

6 . 

7. 

8. 

9. 

10. 


11. 

12. 

13  . 


Field  Length 

FILE  2:  CYCLE  TIMES  (day  hr  min) 

Time  the  need  for  directive/decision  first  perceived 
Time  warning  order  issued  to  subordinate  command 
Time  directive/decision  communicated  to  subord.  comm. 
Intended  (specified)  execution  time  of  directive 

{Planners  involved 
Options  considered 
#  times  this  directive  queried 
Was  this  cycle  initiated  because  of 
monitoring  error (s)?  no:0  yes:l 


FILE  3:  COORDINATION 

Time  necessity  for  coordination  first  perceived 
Time  of  decision 

Time  of  request  or  report  (999  if  decision  was 
to  do  nothing) 

#  times  this  request  or  report  queried 
Response  to  request  within  desired  time  window? 
no:0  yes:l 


FILE  4:  UNDERSTANDING  TIMES  (day  hr  min 
Time  at  which  understanding  was  expressed 
End  time:  period  of  understanding 
Did  understanding  match  ground  truth?  no:0 

Was  understanding  "not  incorrect"?  no:0 

Did  understanding  include: 

composition?  no:0 

disposition?  no:0 

courses  of  action?  no:0 

combat  power?  no:0 

activity?  no:0 

Did  understanding  cause  initiation 
of  a  control  cycle  no:0 


yes :  1 
yes :  1 

yes :  1 
yes :  1 
yes:  1 
yes :  1 
ves :  1 


FILE  5:  ESTIMATE  TIMES  (day  hr  min) 

Time  estimate  was  made 

Time  directive  was  issued 

End  of  the  time  covered  by  estimate 

Number  of  staff  members  contributing  to  the  estimate 
Number  of  options  considered 
Did  estimate  include: 

§  rediction  of  enemy  reaction?  no:0  yes:l 

egree  of  mission  accomplishment?  no:0  yes:l 

residual  capacity  of  friendly  units?  no:0  yes:l 

residual  capacity  of  enemy  units?  no:0  yes:l 

Was  estimate  correct?  :1 

not  incorrect?  :2 

incorrect?  :3 

Number  of  assignments  in  this  directive 
Number  of  assignments  which  contradict  the 
assignments  of  the  Commander's  decision 
Number  of  queries  about  this  directive 


1 

1 

1 

1 


1 

2 

1 

2 


Figure  3.  Additional  ACCES  data  files. 
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recording  not  raw  location  data  but  non-classif ied  differences. 
Individual  plotting  discrepancies  could  be  retained  in  a  separate 
Monitoring  file  with  mean  errors  recorded  in  File  1. 

Follow-on 


For  ACCES  applications  to  be  carried  out  beginning  in  1990  we 
anticipate  a  gradual  clarification  and  "hardening"  of  the  vari¬ 
ables  in  order  to  eliminate  as  many  of  the  ambiguities  as  pos¬ 
sible.  Rather  than  using  summary  percentages  for  investigating 
effects  and  relationships,  more  elemental  counts  will  be  uti¬ 
lized.  For  example,  instead  of:  the  percent  of  planning  cycles 
for  which  two  or  more  staff  members  participated  in  the  develop¬ 
ment  of  alternative  courses  of  action,  we  will  have:  the  number 
of  staff  members  who  participated  in  each  planning  cycle.  From 
these  numbers  it  will  be  possible  to  make  a  distribution  showing 
how  many  planning  cycles  involved  one  staff  member,  how  many  two, 
how  many  three,  etc.  It  may  be  that  two  or  three  staff  members, 
working  together,  are  able  generally  to  formulate  more  complete 
and  accurate  estimates  than  a  single  person  working  alone.  How¬ 
ever,  it  may  be  only  under  fairly  circumscribed  conditions  that 
they  can  do  this,  therefore,  we  will  want  to  be  able  to  examine 
these  counts  according  to  prevailing  conditions.  Further,  it  may 
be  that  under  certain  circumstances  it  is  counter-productive  to 
have  more  than  one  (or  three,  or  soT.^  ocher  number)  staff 
member(s)  developing  alternative  courses  of  action.  We  may  be 
able  to  find  out  if  this  is  urue  and  if  it  is,  be  able  to  specify 
some  of  the  circumstances  for  which  it  is  true. 

It  is  recognized  by  many  that  there  are  so  many  factors  which 
contribute  to  battle  outcome  that  it  cannot  serve  as  a  very  sen¬ 
sitive  criterion  against  which  to  evaluate  command  and  control. 
Under  the  assumption  that  commanders  and  their  staff  members, 
by  virtue  of  their  selection,  training,  and  experience,  are  not 
only  capable  of  judging  but  in  the  best  position  to  judge  C2 
operations  in  which  they  are  involved,  a  very  brief  and  direct 
questionnaire  (see  Appendix  A)  has  been  developed. 

During  a  CPX  at  which  ACCES  is  being  applied,  the  commander 
and  selected  members  of  his  staff  could  be  asked  to  fill  out  this 
questionnaire  approximately  two  times  a  day.  The  rebarbative 
nature  of  this  kind  of  task  is  minimized  by  the  brevity  of  this 
one.  It  should  take  no  more  than  two  minutes  to  complete.  .  . 
less  as  it  becomes  more  familiar.  If  we  do  this  and  there  is 
substantial  agreement  among  these  experts,  their  responses  will 
be  combined  and  used  to  elucidate  ACCES  measures.  If  there  is 
wide  diversity  of  responses  we  could  try  to  understand  why  and 
see  if  there  is  any  information  in  the  responses  which  can  help 
to  clarify  the  nature  of  the  ACCES  measures.  More  likely  would 
be  results  between  these  two  extremes  such  as  similar  views 
within  a  CP  which  differ  from  those  of  another  CP  or  similar 
views  within  an  echelon  but  differing  views  between  echelons. 
However  that  turns  out,  the  questionnaire  has  a  high  degree  of 
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face  validity  and  relations  between  the  responses  to  these 
questions  and  many  of  the  ACCES  measures  would  help  to  establish 
ACCES  construct  validity. 

Summary  and  Conclusions 

The  analyses  conducted  on  the  data  obtained  from  the  ACCES 
applications  studied  have  provided  preliminary  insights  into  the 
general  strengths  and  weaknesses  in  command  and  staff  processes. 
The  divisions  studied  performed  well  in  their  ability  to  involve 
multiple  staff  members  in  the  formulation  of  clear  and  comprehen¬ 
sible  plans  in  accordance  with  their  commander's  expressed  in¬ 
tent,  and  to  disseminate  them  in  a  timely  manner.  Conversely, 
their  performance  was  relatively  weaker  in  the  areas  of  thorough¬ 
ness  of  briefings  and  accuracy  with  which  forces  were  plotted  on 
command  post  maps.  These  trends  cannot  be  construed  as  absolute 
truth  inasmuch  as  the  sample  size  used  in  the  analyses  was  small, 
and  ACCES  is  imperfect  as  a  measurement  methodology.  In  addition 
to  the  purpose  of  exploring  the  possibilities  in  regard  to  the 
types  of  questions  and  problems  for  which  the  ACCES  methodology 
could  provide  assistance  in  finding  solutions,  the  present  pro¬ 
ject  was  an  attempt  to  examine  the  utility  of  the  measures  and 
propose  fixes  where  possible. 

As  a  result  of  the  analyses  conducted  in  this  project,  it  has 
become  apparent  that  the  most  debilitating  problem  with  the  ACCES 
methodology  is  the  reliance  upon  percentage  figures.  For  most  of 
the  uses  in  which  the  methodology  could  be  applied,  raw  scores  or 
frequency  counts,  rather  than  percentages  would  provide  a  great 
deal  more  precise  and  reliable  information.  Another  of  the  prob¬ 
lems  noted  with  the  measures  is  the  unrealistic  stringency  with 
which  monitoring  and  completeness  are  measured.  Recommendations 
have  been  made  to  revise  the  criteria  on  these  measures  to  more 
realistically  reflect  performance. 

ACCES  has  been  applied  during  six  division  exercises  in  ap¬ 
proximately  18  months.  This  is  a  very  short  time  in  the  develop¬ 
mental  lifespan  of  a  complex  measurement  technology,  and  the 
measurement  of  command  and  control  is  certainly  complex.  Over 
this  time,  many  changes  were  made  to  the  measures  as  well  as  to 
the  data  collection  and  analysis  techniques.  This  evolutionary 
process  has  resulted  in  many  improvements.  However,  in  this 
review  we  have  found  several  examples  of  ACCES  measures  which  are 
still  less  than  satisfactory.  Development  of  improvements  in  the 
measures  and  procedures  is  the  task  of  a  follow-on  contract 
scheduled  to  begin  early  in  calendar  year  1990.  This  review  has 
also  found  evidence  of  a  methodology  which  shows  great  promise 
for  becoming  highly  useful,  not  only  for  its  primary  and  obvious 
purpose  but  also  for  extensions  of  that  purpose.  In  addition  to 
providing  general,  overall  measures  of  C2  at  the  division  and 
corps  echelons,  ACCES  will  make  it  possible  to  evaluate  physical 
and  operational  changes  in  the  command  post  environment.  Feasi¬ 
bility  has  been  demonstrated. 
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Appendix  A 


f 


COMMAND  AND  CONTROL  QUESTIONNAIRE 

The  following  questions  are  designed  to  obtain  vnur  opinion  of  the  quality  of  performance  of  certain  aspects 
of  the  command  and  control  system  for  the  past  12  hours.  Please  answer  to  the  best  of  your  knowledge  and  from 
your  own  point  of  view.  Each  question  is  accompanied  by  a  line  marked  from  zero  to  100%.  The  zero  (left) 
end  of  the  line  is  for  extremely  poor  performance  and  the  100%  (right)  end  is  for  extremely  good  or  perfect  per¬ 
formance.  Place  a  single  mark  on  the  line  at  die  position  which  indicates  your  opinion. .  .toward  the  left  for  worse, 
toward  the  right  for  better. 

How  well  has  C2  performed  the  following  functions? 


1.  Fulfill  priority  intelligence  requirements; 


0  10  20  30  40  50 

2.  Translate  commander’s  intent  into  action: 

60 

70 

80 

90 

100% 

0  10  20  30  40  50 

3.  Inform  commander  of  progress  of  the  battle: 

60 

70 

80 

90 

Too% 

0  10  20  30  40  50 

4.  Dissemination  of  orders  to  lower  echelons: 

60 

70 

80 

90 

1oo% 

0  10  20  30  40  50 

5.  Synchronization  of  entire  division  effort: 

60 

70 

80 

90 

“Too% 

0  10  20  30  40  50  60 

6.  Appropriate  delegation  of  authority  by  senior  commanders: 

70 

80 

90 

1oo% 

0  10  20  30  40  50 

7.  Engineer  planning  support  of  commander's  intent: 

60 

70 

80 

90 

1oo% 

0  10  20  30  40  50 

8.  Fire  planning  support  of  commander’s  intent: 

60 

70 

80 

90 

~~Too% 

0  10  20  30  40  50 

9.  Coordination  of  combat  service  support: 

60 

70 

80 

90 

~loo% 

0  10  20  30  40  50  60  70  80 

10.  Lateral  coordination  among  the  division  staff  elements  (Gl,  G2,  G3,  etc.): 

90 

~~Too% 

0  10  20  30  40  50  60  70  80 

1 1.  Horizontal  coordination  among  division  CPs  (DMA1N,  DTAC,  DREAR): 

90 

”Too% 

0  10  20  30  40  50 

60 

70 

80 

90 

100% 

SUMMARY  QUESTIONS: 

A.  During  the  last  12  hours,  to  what  degree  was  there  successful  progress  toward  mission  accomplishment? 

0  10  20  30  40  50  60  70 

B.  What  is  die  capability  of  the  division  for  continuing  the  mission? 

80 

90 

100% 

0  10  20  30  40  50  60  70 

C.  What  is  the  enemy’s  capability  for  continuing  their  mission? 

80 

90 

100% 

0  10  20  30  40  50 

60 

70 

80 

90 

100% 
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